Skip to content

fix(ai): log roast-me stream failures and pass system prompts via system - #35

Merged
lautaropaske merged 1 commit into
mainfrom
fix/ai-sdk-error-logging
Oct 2, 2026
Merged

lautaropaske merged 1 commit into
mainfrom
fix/ai-sdk-error-logging

Conversation

@lautaropaske

Copy link
Copy Markdown
Contributor

Summary

Two fixes from the Dash0 log review:

  1. roast-me stream failures are now logged. streamObject keeps errors inside the stream, so a failed model call or a schema mismatch returned 200 and logged nothing.
  2. System prompts go through the system parameter instead of messages. This removes the AI SDK prompt-injection warning that made up 1,256 of about 1,280 WARN logs in the last 14 days.
 streamObject / generateObject / generateText({
-  messages: [{ role: "system", content: prompt }, ...rest],
+  system: prompt,
+  messages: rest,
+  onError: ({ error }) => console.error("[roast-me/analyze] stream failed", error),  // roast-me only
 })
Route File
/api/grade src/pages/api/grade.ts, src/resume-checker/prompts/grade.ts (messages() drops the system message and its parsed arg)
/api/analyze-take-home src/takehome-checker/index.ts
/roast-me/api/analyze src/app/roast-me/api/analyze/route.ts

roast-me has the repo's only streamText/streamObject call. OpenAISdkAIClient (behavioral-checker) is unchanged. It uses the raw OpenAI SDK, which has no system option and doesn't emit this warning.

Evidence

The same three requests ran against a local dev server with no AI keys, so every model call fails with an auth error.

  • Before:
    AI SDK Warning: System messages in the prompt or messages fields can be a security risk ...
     POST /roast-me/api/analyze 200        ← failure not logged
    AI SDK Warning: System messages in the prompt or messages fields can be a security risk ...
     POST /api/analyze-take-home 500
    AI SDK Warning: System messages in the prompt or messages fields can be a security risk ...
     POST /api/grade 500
    
    After:
    [roast-me/analyze] stream failed [Error [GatewayAuthenticationError]: Unauthenticated request to AI Gateway.
     POST /roast-me/api/analyze 200
    Error [AI_LoadAPIKeyError]: OpenAI API key is missing ...
     POST /api/analyze-take-home 500
    Error [GatewayAuthenticationError]: Unauthenticated request to AI Gateway ...
     POST /api/grade 500
    

bun tsc passes.

Merge Danger

Door: two-way

Revert the commit to undo it.

Blast Radius: alerts

The Dash0 rule exists error in production @ vercel (open-silver) fires on any ERROR log, so real roast-me failures now reach Slack. That is the goal. The prompts are identical, only where they are passed changed, so grading output should stay the same. The happy path was not run with real keys.

🤖 Generated with Claude Code

…stem`

roast-me's streamObject had no onError, so model and schema failures
never reached the logs. Grade, take-home and roast-me put the system
prompt in `messages`, which triggered the AI SDK prompt-injection
warning on every request (~98% of open-silver WARN logs in Dash0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
open-silver Ready Ready Preview Oct 2, 2026 7:11pm UTC

Request Review

@lautaropaske
lautaropaske merged commit 1ddfe14 into main Oct 2, 2026
3 checks passed
@lautaropaske
lautaropaske deleted the fix/ai-sdk-error-logging branch October 2, 2026 19:18

This branch was successfully deployed

1 active deployment
Preview — f55e6e2f Deployed Oct 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant